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ABSTRACT 

A year- long ethnographic case study ex A timed twc 
public city high school English teachers" efforts to reform their 
literature instruction and evaluation practices through the use of 
portfolios. One case study teacher ("William") has 30 years of 
teaching experience and was heavily influenced by New Criticism. The 
other teacher ("Leslie") had been teaching for one~and-a-hal f years 
and believed that literature learning involves making connections 
between the text and personal experience. Five focus students in each 
class were identified. Classrooms were observed and teachers were 
interviewed. Throughout the school year. William rarely deviated from 
"evaluative" readings of students' work. However. Leslie's response 
framework shifted as she struggled to balance her role as a 
'"gatekeeper" with her role as "coach." Both teachers had difficulty 
discussing the focus students' literature learning in terms of 
specific patterns of growth even though students' portfolios were 
available for them to examine. Findings reveal the vulnerable 
position of English teachers who are unsure how to evidence success 
in their teaching and in their students' learning. This is especially 
true with the current push for greater accountability in public 
education. (Contains 12 references.) (RS) 
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AERA ANNUAL CONVENTION, APRIL 1994, NEW ORLEANS, LA 

ROUNDTABLE SUMMARY 
Assessing Literature Learning: Teachers' Response Frameworks and 
Descriptions of Students 1 Progress 
Mary H. Sawyer, University at Albany, SUNY 

CO 
CO 

Q Description and Objectives: This presentation is based on a 

year-long ethnographic case study of two English teachers' 
efforts to reform their literature instruction and evaluation 
practices through the use of portfolios. The teachers taught at 
a public city high school with a culturally diverse student 
population and were members of a teacher- research group called 
the Portfolio Assessment Project, This 3-year university-school 
collaboration involves 13 teachers, two research assistants (of 
which I am one), and is directed by Alan Purves at SUNY-Albany. 
The teachers work as primary researchers, developing their own 
portfolio research questions and projects; university-based staff 
work to support teacher- researchers (e.g., arranging bimonthly 
Project meetings, providing articles on portfolios, observing 
their classrooms, encouraging their efforts). The presentation 
is based on the iirst year of the Project. 

The ^eacarch tlosign is a qualitative, nested case study 
where each ol t hi? two U?«ichers and their classrooms served as a 
case, with five tocus students embedded within each case. One 
case-study teacher ("William") has been teaching 30 years; his 

— instructional goals were heavily influenced by his background in 
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New Criticism and his belief that students should heed Ciardi's 
advice and learn to read literature "for what it says, not for 
what you think it says." The other teacher ("Leslie") has been 
teaching 1 1/2 years; her instructional goals were informed by 
her belief that literature learning involves making connections 
between the text and personal experience. The teachers each 
elected their ninth-grade (middle track) classes as their 
research focus. 

Perspectives: At the heart of the English curriculum is 
literature: at the secondary level, literature dominates some 
50-78% of English class time (Applebee, 1990, p. 48), while at 
the primary level, literature is becoming a more prominent 
feature of the curriculum, displacing somewhat the reliance on 
basal readers in the teaching of reading (Langer & Allington, 
1992). Yet despite literature's growing prominence in the 
curriculum, the goals of literature instruction remain diffuse, 
with no clear agreement as to what we want students to know once 
they "comprehend" a text. Traditionally "knowing literature" has 
meant being familiar with literary terms, canonic texts, standard 
interpretations, and (in upper-track classes) textual analysis- 
knowledge that could be evaluated by multiple-choice texts or 
short essays. Yet these marks of knowing have been criticized 
for their lack of emphasis on students' thinking. 

In the past 20 years a growing body of work from a variety 
of disciplines has led many scholars and educators to broaden our 



understanding of comprehension, arguing that knowing literature 
involved more than a static rendering; comprehension is described 
as "meaning-making," the process by which readers create a "poem" 
from the literary text (Rosenblatt, 1978), Response as a mark of 
knowing supports students' understandings and meaning-making 
processes, but there is little understanding as to how to 
document progress in students 1 meaning-making, and, indeed, what 
this progress looks like or even if it is likely to be evident in 
the course of a school year. Because response can be anything 
from journal entries to drawings to students* own fiction, and 
because a text's difficulty is determined by a variety of factors 
and contexts, the task of documenting literature learning over 
time can be quite complex. 

Only very recently have researchers begun to reconceptualize 
evaluation in a way that makes it compatible with these dynamic 
and complex views of knowing (Wolf, Bixby, Glenn, & Gardner, 
1991). The role of evaluation has been broadened, from meiiily 
measuring learning to supporting it and shaping instruction 
(Calfee & Hiebert, 1988); the focus of literacy evaluation has 
shifted from the identification of deficits in students' learning 
to providing "room" for students to voice their understandings 
(Genishi & Dyson, 1984). In addition, evaluation has come to be 
recognized as an interpretive, cultural act (Gilmore, 1987; 
Johnston, 1992) that is set in a dynamic relationship with 
instruction and teachers' beliefs about learning (Anson, 1989; 
Phelps, 1989; Shaughnessy, 1976). In the language arts, 
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portfolios are quickly becoming one of the most popular of these 
new assessment methods with many states, schools, districts, and 
teachers designing and implementing their own versions. Yet 
underlying the current enthusiasm for portfolios lies a concern 
that this fervor will quickly wane and that porttolios will end 
up being "about change without difference" (Koumor, 1991, p. 
447) . 

Methods: During the 1992-1993 academic year I observed the three 
literature units (at the beginning, middle, and end oi the year) 
in each of the two classrooms. During the observations I took 
fieldnotes and audiotaped classroom interactions, paying 
particular attention to teachers' oral and written responses to 
student work. I interviewed the teachers before and after each 
unit (and had on-going discussions with them about their work) 
and interviewed focus students at the middle and end of the year. 
Student work and portfolios (along with any teacher comments or 
marks) were photocopied. The teachers and 1 also discussed their 
portfolio plans and practices with the other teacher-researchers 
at bimonthly Project meetings; these meetings and other informal 
discussions were audiotaped. 

These data were analyzed by nu>ans of analytic induction and 
constant comparison in which data were col leoted and analyzed 
simultaneously and emerging themes continually checked against 
incoming data and then modi Lied and refined. 
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Results: 

Teachers [ response frameworks . Throughout the school year 
William rarely deviated from "evaluative" readings of students' 
work (cf., Phelps, 1989). Thus, his oral and written responses 
to students mostly involved making judgments and noting deficits. 
William felt his expertise as a teacher lay in his role as an 
judge: "This is why they are paying me sixty-some thousand 
dollars a year." 

In contrast to William, Leslie's response framework shifted 
and changed during the school year as she struggled to balance 
her role as a "gatekeeper" with her role as "coach." While she 
had always tried to read student work " f ormatively, " (cf., 
Phelps, 1989) seeing students' words and texts as evolving 
drafts, she also believed at the beginning of the year that 
students would feel "rootless" without her grading their work and 
hence without her evaluative readings. She felt evaluative 
readings were "fair" as long as she had specifically taught the 
skills she graded, although she realized this fairness might 
adversely effect their learning ("If I say, 'A piece has to have 
"X" amount of detail' . . . what am I doing to their own sense of 
how to write or think, or to come up with ideas and structure 
things for themselves?") 

As Leslie continued to reflect on evaluation issues, she 
grew more uncomfortable with her evaluative readings , arguing 
that grades did not "open up new understandings." Her 
reflections on grades and judgments led her to begin to read 
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student work "contextually , " (cf., Phelps, 1989) seeing student 
texts and her own judgments as situated and embedded in contexts. 
Thus, she began to view grades as one person's "momentary 
impressions" and to realize how texts were shaped by students' 
lives . 

Teachers' understandings of students' literature learning . 
Both teachers had difficulty discussing focus students' 
literature learning in terms of specific patterns of growth even 
though students' portfolios were available for them to examine. 
William's talk focused on students' attitudes and behaviors, such 
as Scott's difficulty "focusing on any sort of directions"; 
Trisha's self -admitted "laziness"; and Craig's 
"conscientiousness." Leslie's descriptions focused more on 
students' relationships to their work (their "commitment" or 
"involvement") and how these attitudes changed or didn't. Yet 
Leslie was unsure whether qualities that she valued, such as 
Arnold's "open and curious mind," were simply a part of students' 
personality or if these qualities had developed as a result of 
the class. She also had difficulty discussing the literature 
learning of a student like Rita, who from the first day of class 
revealed herself in literature discussions to be "very insightful 
and very mature* M Furthermore, she was unsure how to value Jim's 
learning who was "becoming more of a participant in class" but 
whose work was only "average" when compared to others'. 

Significance of Work to the Field: Currently there is no clear 

6 

< 



agreement as to what we want students to know once they 
"comprehend" a text nor is there an articulated understanding of 
what marks growth in literature learning. With the current push 
for greater accountability in public education, this study 
reveals the vulnerable position of English teachers who are 
unsure how to evidence success in their teaching and their 
students ' learning . 
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